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EXECUTIVE SUMMARY 

The federal government funds a significant proportion of education research in the United States. But 
these research efforts have, by and large, fallen short of delivering on the promise to drive individual 
student outcomes across the country. By altering the priorities of education research to embrace a more 
complete research cycle, the federal government can boost the productivity of future efforts. 

Why is education research critical to student success? 

• Systemic upgrading. The vast majority of U.S. schools operate in a century-old, factory-based 
education model that standardizes the way students are taught. Meaningful research is vital in 
enabling schools to transition to newer, more effective learning models that customize to students’ 
distinct needs. 

• Technological advantages. With the rise of technology, education is undergoing a sea change. 
Proper research will help guide the implementation of new tools and resources that will help 
educators optimize the learning experience for each student. 

• Empowered educators. Educators face complex circumstances, like what to do when a child is 
struggling or how best to engage and motivate students. Research is critical in arming educators 
with useful information about how best to serve each student. 

• Increased proficiency rates for all students. Today’s standardized factory model of education 
yields highly variable student outcomes. The past decades have been fraught with only gradual 
progress toward closing persistent achievement gaps. Better research can help the system improve 
more quickly and predictably against these goals. 

What is the current state of education research? 

• Funding is limited and fragmented. The federal government primarily funds education research 
through three agencies: the Institute of Education Sciences, the National Science Foundation, and 
the National Institutes of Health. Education research funding levels remain far below those of 
other industries such as defense, energy, and health care. 


1 



• Randomized controlled trials are the gold standard. Randomized controlled trials, or RCTs, 
involve comparing students who experienced a given approach with those who did not in an 
effort to surface effective practices. President George W. Bush’s administration created the What 
Works Clearinghouse, which promulgated these trials as a key method to unearth successful 
teaching practices. 

• RCTs unearth what is most likely to work, but do not offer precise guidance. RCTs are often 
expensive and time-consuming, and they typically only shed light on what is most likely to work on 
average, whereas educators need a level of information to chart predictably effective paths for each 
student. For example, a recent federally funded RCT analyzing the efficacy of an adaptive math 
software found that the software led to an average eight-percentile point gain among students. 
Although a thorough, well-funded study like this signals promise, findings like these do not reliably 
tell us why some portion of students or certain classes likely didn’t fare as well whereas others 
fared far better. 

• The research cycle is incomplete. If we want to understand what works predictably for different 
students in different circumstances, then the current research process is incomplete. Although 
RCTs help identify useful practices and are an important step in the research process, when 
treated as an end they rarely provide information that is actionable for decision-makers in the 
education system. We need to develop theories that explain why given experiences or 
interventions do or do not work for given students in different circumstances. 

A research agenda for student-centered learning 

An effective research agenda moves beyond merely identifying correlations of what works on average to 
articulate and test theories about how and why certain educational interventions work in different 
circumstances for different students. To do this, the government will need to: 

1. Focus on the individual, not the average 

• Encourage research that pushes our understanding beyond the average student and instead 
works to discover predictably effective paths for each student. 

• Take advantage of technology-enabled structural shifts to study what works for specific students 
in specific circumstances. 

• Fund efforts that make data collection more seamless and less arduous on districts in order to 
allow schools and researchers to collect better, more real-time data on what is actually happening 
in schools. 

• Support research that progresses past initial RCTs and promotes alternative methods for 
unearthing what drives student outcomes in different circumstances. 
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2. Push toward causal understanding 

• Focus on the development of theories that explain why certain approaches do or do not work in 
certain circumstances. For example, do certain interventions work best for certain student 
populations at a certain point in their learning trajectory? Do particular software tools work better 
for practicing hard skills, whereas others excel at training students to persist through challenges? 

• Encourage research that digs in on anomalies— instances where the prevailing research cannot 
explain a certain result— to surface new explanations and refine our understanding of what drives 
individual learning. 

• Serve as a hub for collecting and studying these anomalies. 

3. Continue to fund research, not development 

• Focus on the research in research and development (R&D) in order to shed light on what works 
for what students in what circumstances— and allow the education industry to develop tools in 
accordance with those findings. 

• Beware of conflating needs for actual market demand. Policymakers and funders often perceive 
needs in the system that do not translate into areas where schools will spend money. As a result, 
the government could fund the development of tools that might not see the light of day in actual 
classrooms. 

4. Coordinate the research cycle toward completion 

• Maximize the use of scarce research funds by creating criteria for the types of questions facing 
education that will further the efforts to personalize learning. To do this, fund core questions that 
support educators on the ground by shedding light on what works, for which students, in what 
circumstances. 

• Deploy research spending to usher in a new, more complete cycle of research; although RCTs 
are important, they are neither the king nor the final step of the research process. 
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WHAT’S WRONG WITH THE NATION’S APPROACH TO EDUCATION RESEARCH? 


As the 2016 presidential election approaches, the next U.S. president will have the opportunity to bolster the 
federal research agenda in education. Historically, research in education has fallen short both in terms of funding 
and strategy. But a coherent research agenda will be vital to bringing America’s schools into the 21st century. 

Over the past decade, President George W. Bush and President Barack Obama have each proposed agendas to 
expand our fundamental understanding of what works in education and to scale the development of tools to help 
drive learning. Under the Bush administration, the creation of the What Works Clearinghouse, the U.S. 
Department of Education's database of high-quality research, formally ushered in the randomized controlled trial, 
or ROT, as the "gold standard” of education research. An ROT is a study design that randomly assigns participants 
to an experimental group or a control group, which, in turn, isolates the effects of the tested intervention. By 
creating a higher bar for determining what research would gain the U.S. Department of Education’s stamp of 
approval, the What Works Clearinghouse continues to attempt to support and highlight research that can answer 
the question "does this intervention work?” with confidence. 1 

The Obama administration focused on increasing the development and use of research and evidence in program 
selection and resource allocation. This effort included defining shared standards of evidence between the 
Institute of Education Sciences and the National Science Foundation on a spectrum ranging from simple data 
gathering on early stage developments to gold standard evaluations; using the Investing in Innovation Fund (i3) 
and other programs to establish a framework for how educational programs should receive funding based on the 
type and level of evidence supporting their efficacy; creating research partnerships between school systems and 
networks of researchers; training researchers and technical assistance providers to develop and manage rigorous 
evaluations; and coding and making open the metadata from tens of thousands of evaluations. Additionally, given 
the U.S. Department of Education’s limited resources, the Administration launched an interagency subcommittee 
on learning science and technology as part of the National Science and Technology Council. This subcommittee 
spawned several complementary efforts— including the BRAIN Initiative. 2 The Administration also unsuccessfully 
attempted to move beyond just funding research by creating an Advanced Research Projects Agency (ARPA) for 
education to pursue an advanced research and development agenda modeled after the Defense Advanced 
Research Projects Agency (DARPA)— the prolific development arm of the U.S. Department of Defense— that would 
fund directed development projects in education. 3 

Both administrations’ efforts took steps toward bolstering the level of rigor in education research and improving 
the research process. Neither, however, went far enough toward giving educators on the ground actionable 
insights that would allow them to serve effectively each and every student in their schools. 

This limitation has become more glaring than ever before, as the rapid rise of technology is changing education as 
we know it. We predict that by 2019, half of all high school courses will be online in some form or fashion. And 
across K-12 education, we have observed and documented the proliferation of blended-learning models that 
weave together online learning and traditional schools. 
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At its best, integrating technology into our schools and classrooms 
stands to mark a radical departure from our traditional, factory-based 
school system that continues to fall short of our aspirations of 
reaching all students. Armed with technology, no longer must we 
resort to “teaching to the middle” or allowing students to progress to 
new material and higher grade levels before they’ve mastered the 
fundamentals. With technology as a tool, schools can move from a 
one-size-fits-none approach to a student-centered one. Teachers can 
gain a far more precise understanding of how individual students are 
progressing and provide them with just-in-time materials and supports 
suited to their needs and strengths. In some schools that are taking 
advantage of the power of online learning, students can chart distinct 
paths that allow them to access, at their own pace, a variety of online 
and offline learning modules, supports, and experiences that are 
tailored to their learning needs. Figure 1 illustrates this sea change 
from a factory-based to a student-centered model. 

In other words, for the first time in our country’s history, technology affords schools a structure capable of 
reaching each student at scale. This raises fundamentally different questions that we need education research to 
tackle. In this new context, neither President Bush’s focus on what interventions are most likely to work, according 
to RCT studies, nor President Obama’s focus on STEM education, innovative development projects, or 
tiered-evidence standards that do not progress beyond an initial RCT will suffice. 

Instead, the ability to predict what works, for which 
students, in what circumstances, will be crucial for building 
effective, personalized-learning environments. The 
current education research paradigm, however, stops 
short of offering this predictive power and gets stuck 
measuring average student and sub-group outcomes and 
drawing conclusions based on correlations, with little 
insight into the discrete, particular contexts and causal 
factors that yield student success or failure. Those 
observations that do move toward a causal understanding 
often stop short of helping understand why a given 
intervention or methodology works in certain 
circumstances, but not in others. 


The ability to predict what 
works, for which students, 
in what circumstances, will be 
crucial for building effective, 
personalized-learning 
environments. 


For the first time in 
our country’s history, 
technology affords 
schools a structure 
capable of reaching 
each student at scale. 


These insights, in turn, fail to drive the effective development of tools that would actually transform student 
outcomes. Technology, in other words, will only be as good as our understanding of what drives learning in which 
circumstances. Technology, of course, can be a tool to develop that understanding through sophisticated data 
analytics and faster research cycles; this will only be the case, however, if we use technology in the right ways to 
answer new questions about how students learn. 

To usher the nation’s schools into the 21st century, the next president should conquer the chronic 
shortcomings of education research and call for a new way of investigating what drives learning. What follows 
summarizes what a new research methodology ought to look like and how the next president can put these ideas 
into action. 
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Transforming K-12 Education 


Traditional Education 

• Industrial Age 

• Mass Production 

• Time Constant & 
Achievement Variable 

• Advance Based on Seat Time 

• Geographically Limited 


(J) Student-Centered Learning 

>/ Knowledge Age 

✓ Customization 

✓ Time Variable & 

Achievement Constant 

✓ Advance Based on Competency 

✓ Geographically Flexible 




EDUCATION IS STUCK IN AN INCOMPLETE RESEARCH CYCLE 


There is a lot of education research. Some is filled with mountains of 
statistical evidence, other consists of the results from an RCT, and still other 
examines various case studies. But the prevailing paradigm in which education 
researchers have been trapped does not give them a chance to produce 
research that can predictably lead to better schools. Instead, the existing 
paradigm causes researchers to stop their work when it still does not 
provide complete answers to practitioners. This research gives us detailed 
statements of correlation, which sometimes control for a wide range of 
variables. But it does not provide us with sound theories of causation that 
researchers can continue to test and refine. Most— although certainly not 
all— education research consequently creates more contention than 
consensus. Even the What Works Clearinghouse, which sets parameters for 
evidence standards to improve the effectiveness of various education 
interventions, provides insights that are at best incomplete. Although this 
effort sets higher standards for the sorts of evidence that constitute "proof” 
of a particular intervention’s effectiveness, the findings featured in the What 
Works Clearinghouse fail to communicate whether given interventions will 
work for teachers and administrators working with individual students in 
different circumstances across the numerous local contexts that make up our 
public education system. 




The ability to know what actions will lead to desired results for a specific 
student, in a specific school, in a specific situation, awaits the 
development of a more iterative research process 5 Figure 2 illustrates 
this more complete research cycle. 


Moving to predictive, circumstance-based theories of causation in 
education will require a shift beyond the prevailing paradigm. No longer 
will research on "best practices” or “what works” best on average — or 
what is most likely to work for any student— across education suffice. 
RCTs can be useful preliminary steps on the road to robust bodies of 
understanding, but when education research reaches this stage, it all too 
often fails to progress beyond it. This stage causes paralysis when 
educators cannot determine from such studies whether following an 
average formula will lead to hoped-for outcomes for specific students. 
Moreover, empirical knowledge may be good at explaining the past, but 
only a causal understanding can help us peer into the future. 
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Life of the Research Cycle 


Where the current research cycle ends 


Observe 

interventions 


Researchers 

observe 

phenomena that 
appear to be 
driving or failing to 
drive student 
outcomes. 


Categorize 
& test 


Researchers 
hypothesize about 
what specific 
factors may be 
causing an increase 
or decrease in 
student outcomes 
and test them 
through 

qua si -experimental 
studies, A/B tests, 
or randomized 
control trials. 


Propagate 
results 
“on average’' 


Researchers 
publish the results 
of these studies— 
often as academic 
articles— that 
explain the 
effectiveness on 
average of a given 
intervention. 


Observe 
anomalies & 
sort by 
circumstance 


Researchers dig in, 
often through a 
series of "n of 1 " 
studies or other 
methods, to 
understand 
anomalies to their 
findings and study 
the specific 
circumstances in 
which their results 
did not accurately 
predict what 
actually occured. 


Refine theory 
of causation 


Researchers revisit 
categories in the 
original research to 
refine their theory 
of causation and 
test these new 
Hypotheses. 


Figure 2 


In traditional education research, researchers begin by observing phenomena, such as identifying situations in 
which students are succeeding or failing to varying degrees. They then move on to define and test the categories 
that they think contribute to those outcomes, often through quantitative research that tests the effectiveness of 
various interventions. This phase of the research is focused primarily on describing interventions and correlating 
results and sometimes involves randomly allocating treatments and comparing treatment and control groups. The 
RCT can be a vital tool at this stage of research. In addition to random assignment, in many cases, researchers can 
use sophisticated modeling to test mediating and moderating variables along with overall significance in 
outcomes. And increasingly, these studies also include latent class analysis and within-group measures to study 
effect sizes specific to certain student populations or contexts. Such studies help us gain a better understanding 
of how much a particular treatment did or did not influence the outcome of interest, thereby moving researchers 
toward an initial understanding of causality. 

Alternatively, as a cheaper and quicker approach, researchers or practitioners can use A/B testing— common today 
in marketing— to test their hypotheses in lower stakes environments. Many times shorter cycle A/B tests can start 
to reveal what is or isn’t working in a specific classroom to help researchers and educators alike. 

The research process typically ends there, however. This is unfortunate because even the most sophisticated 
statistical modeling will rarely lead to a foolproof understanding of causation. In other words, this sort of modeling 
is built to weed out anomalies— outcomes that the findings of the study cannot explain— as statistical noise, 
whereas understanding these occurrences is critical to discovering causality that leads to actionable insights 
based on different circumstances. As a result, the findings from these elaborate studies tend to focus on the 
average effects at the group or sub-group level of the originally conceived categories, which leave us with little 
actionable insight into the underlying cause of student achievement or lack thereof for many students . 6 


A number of key steps must follow to improve upon 
these preliminary empirical research tests and to help us 
reach an understanding of the causal factors in different 
circumstances that drive learning and provide 
prescriptive— rather than merely descriptive— guidance 
for educators. This must involve progressing to assert a 
statement of what causes the outcome of interest— 
a theory, in other words. Researchers must then put on 
their statement of causality like a set of lenses and cycle 
deductively back to observe, categorize, and test their 
theory across a variety of contexts. They say, "When we 
observe these actions being taken, these should be the 
outcomes we observe." 

During this phase, researchers must continue to refine statements of causation on the basis of new forms of 
inquiry. First, they can conduct more narrow "n of V studies of particular individuals within the original study to 
capture nuances that statistical analyses may have missed. Second, they can seek exceptions, called anomalies, to 
the average tendencies identified in their initial empirical work. It is only when we find something that the 
preliminary empirical research’s correlation cannot explain that the research can improve. Such new insights and 
anomalies are good news because they allow researchers to say, "There’s something else going on here,” and that is 
what leads to better understanding. They then must delve back into the initial steps of the research cycle and 
account for the anomalies by revisiting the categorization stage and continuing to test their theory from there. This 
work occurs rarely in education research— or if it does, it does so in a way that does not improve existing 
theories or evidence, but is instead framed as directly contradictory to prevailing findings. 

There is another significant difference here that departs from the traditional research paradigm in education. Rather 
than categorizing by different attributes of the phenomena or different demographics of human subjects, 
researchers building toward an understanding of causation start to categorize the different circumstances in which 
administrators, students, or teachers might find themselves. They do this by asking, when they encounter an 
anomaly, "What was it about the situation in which those people found themselves that caused the causal 
mechanism to yield a different result?" If they find themselves in a circumstance where they must change actions 
or organizations to achieve the desired outcome, then they have crossed a salient boundary into another category. 
As they cycle through this research process, anomaly-seeking researchers will ultimately define each of the 
different circumstances in which administrators, teachers, or students might find themselves when pursuing the 
outcomes of interest. 

Unfortunately, research into programs designed to personalize learning rarely go far enough to unearth such causal 
mechanisms or circumstance-based insights. In addition to random assignments to condition, research of this sort 
will also require better set-ups, better data about participants, and better analytics to tease out what works for 
whom, as opposed to simplistic reporting only on effect sizes for whole populations. Without the complete 
research cycle outlined above, we stand to limit what practitioners looking to personalize learning to their students’ 
needs and circumstances can actually do with the research findings. 


New insights and anomalies 
are good news because 
they allow researchers to 
say, “There’s something 
else going on here/’ and 
that is what leads to 
better understanding. 




For example, a recent U.S. Department of Education-funded RAND RCT study of Carnegie Learning’s Cognitive 
Tutor Algebra I (CTAI) product found that CTAI boosted the average student’s performance by approximately eight 
percentile points. Researchers emphasized that this study considered "authentic implementation” settings— that is, 
they tested the relative effectiveness of the program in actual school settings across a diverse array of students 
and teachers. Carefully studying authentic implementation sounds promising, but only if the researchers can spend 
sufficient time observing and revisiting the authentic circumstances surrounding the implementation, which may be 
affecting student results. Instead, however, when a thorough, well-funded study like this demonstrates that a high 
enough proportion of students benefit from an intervention, we tend to double down on those promising signals. 

In turn, the fact that some portion of students or certain classes likely didn’t fare as well— and others fared far 
better— is treated as probabilistic noise from which statistically significant signals of efficacy must be isolated. But 
generalizing research findings like this will not be helpful if we are trying to build systems that predictably support 
students based on their specific needs. 

To establish a sound theory of precisely why and how a product, such as CTAI or any particular model, can drive 
outcomes, additional stages of research must embrace the descriptive and prescriptive processes outlined above. 
To push these findings further, researchers should dig in on the anomalies hidden within averages— students or 
schools for whom the intervention was not successful or was wildly successful— to tease apart what was different 
about the circumstances where those anomalies arose. With this information, schools and teachers could better 
understand which students, in which circumstances, would benefit from the product or intervention and to what 
extent— and the company itself could improve its offering with additional approaches or a sounder understanding 
of when to use which approach for a wider array of circumstances. With enough information of this sort, school 
systems would have the right information to deploy the right tools that could predictably serve certain students. 

Particular products are not the only research subjects vulnerable to getting stuck in merely descriptive conclusions. 
Even well-known research that withstands the test of time, such as Benjamin Bloom’s “2 Sigma problem” research, 
falls into a similar trap. There is now widespread agreement on the potent effect that Bloom found that 
one-on-one tutoring could have on student learning, including through RCTs; and Bloom’s subsequent studies of 
"objects of change” leant additional insights into the discrete activities and categories within tutoring that showed 
strong effects. Yet, there is still limited understanding of which aspects of one-on-one tutoring actually cause 
different students to reap the benefits of that experience. Not surprisingly, then, even as adaptive courseware 
companies seek to replicate the virtues of the one-on-one tutoring that Bloom wrote about, none have been able 
to achieve foolproof outcomes for all students. 

THE MYTH OF A CLEAR-CUT CUSTOMER IN EDUCATION 

In addition to the incompleteness in the current research paradigm, the education market presents challenges 
when it comes to developing products based on the research to solve endemic problems in schools. Schools will, 
of course, need new tools to differentiate learning with far greater precision than they can today, but 
government-led efforts to move beyond just funding research and into supporting development at this stage— like 
the failed effort to create an ARPA-ED modeled after the military’s prestigious DARPA —pose two risks. 
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First, the old adage "if you build it, they will come" does not pan out in the education market. Demand in education 
is highly fragmented, and factors— such as lengthy procurement cycles, political pressures in different localities, an 
emphasis on things beyond strict academic results, and tight budgets, to name a few— mean that schools’ “needs” 
do not translate predictably into "demand” in the education marketplace. Even gold standard products have 
struggled to see their solutions translate into sales. The elusive customer base in education makes 
government-coordinated development projects a tricky proposition. 8 Second, because districts receive funds from 
so many disparate sources at the state, local, and federal levels with various strings attached and because all of 
these sources have different local political considerations at hand, in practice it is quite difficult to predict demand. 
This makes public school districts— the customers for the vast majority of educational products and services in this 
country— markedly different from the military, which is also government funded but only at the federal level. It 
therefore suggests that an ARPA-ED might not enjoy the success that DARPA has. Similarly, the reason that 
companies don’t invest lots of dollars in basic R&D in education isn’t because they don’t want to fund risky 
projects per se— one rationale for creating ARPA-ED— it’s more because the market won’t support that activity. 

Because of these market factors, federal attempts to contribute directly to the supply side of the education market 
may not hold as much promise as we might hope. For example, the Office of Science and Technology Policy is 
currently spearheading an effort to generate high-impact learning technologies through the use of "pull 
mechanisms”— grants structured to incentivize private sector engagement and competition by creating viable 
market demand for specific products. The traditional design of the XPRIZE is a good example of a pull mechanism, 
as are government challenge grants and social impact bonds. Although this design solves the customer and 
demand problem in the short term, there’s no saying that the products or services that result from the policy will 
be viable in the long run. 

The federal government should resist taking aim at supposedly market-oriented development projects— the kind 
that companies won’t fund because there is no market. The federal government is immune to market pressures, 
but it’s also notoriously bad at predicting school district demand itself. This means that the federal government 
may fund the creation of some exciting things, which may very well not get adopted widely and therefore will 
transform little about the education system. 

A NEW VISION OF RESEARCH: BETTER INFORMATION TO DRIVE PERSONALIZED LEARNING 

With these factors in mind, the government should double down on funding more basic and applied research 
through the entirety of the research cycle described in Figure 2. This would provide much-needed information 
and insights to practitioners at all levels of the education system, particularly to those pursuing personalized 
learning models 9 

The U.S. Department of Education could help to ensure that researchers are posing key questions that will further 
personalized learning. It could also serve as a hub for collecting and studying anomalies to previous studies’ 
findings that need continued refinement! 0 Finally, it could establish more opportunities for circumstance- and 
theory-based research by funding the creation of infrastructure systems that alleviate the friction and costs of 
schools and researchers to collect more and better information on what is actually happening in schools. 

A richer base of theories of causation in education research could also dramatically enhance development efforts 
at less cost to the government by relying on the current surge in private investment in education technology. 
Currently, education companies are conducting development projects in-house, but by and large seem to be doing 
so on the basis of intuition or paltry research that lacks predictive power. As a result, their efforts may have only 
modest effects on student learning. A more robust effort to fund and coordinate more complete research cycles, 
however, would, in turn, arm companies with the right information around which to design new products and 
services that could be effectively personalized to meet students’ needs. 
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That is not to say that the government should never play a role in directed development that fills gaps in the 
supply of education tools. More robust information may also get closer to resolving some of the fragmentation in 
school system demand, as school systems gain reliable insight into what works for what students and deploy 
resources accordingly. Once schools have more precise information about the products at their disposal, then 
patterns of school district and state demand may become more predictable and focused around adopting products 
and services that work in discrete circumstances. Only then will federally supported development projects be a 
potent instrument to yield breakthrough developments in personalized learning tools if the private sector does not 
step up. 

This proposal represents a significant upfront investment in education research, which we have not seen in recent 
decades 11 and remains an acute challenge. Taking the long view, however, suggests that insights into causal 
mechanisms of what drives learning in what circumstances ultimately stands to mitigate the otherwise guaranteed 
waste involved in interventions that prove out on average, but with which we inevitably apply too broad a brush. 
Once we can accurately predict whether something will succeed or fail and in what circumstances, education 
dollars will stand to go markedly further in meeting the needs and capitalizing on the strengths of each student. 


12 


Notes 
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Robert Temple, "Enrichment Design Studies Should Enhance Signals of Effectiveness,” U.S. Food and Drug Administration, December 12, 
2014, http://www.fda.gov/Drugs/NewsEvents/ucm295054.htm (accessed September 9, 2015). 

6. In recent years, a call for more, quicker cycle RCTs has grown in the education reform community. Virtual environments may be especially 
fertile ground for faster cycle experiments of this type. We believe, however, that simply doing more RCTs more quickly may not get us closer 
to causal statements regarding what works in which circumstances. This requires the additional steps, outlined in this paper, of generating a 
theory of causation and testing this theory across a variety of circumstances to refine that theory over Erne. 

7. DARPA essentially works by funding projects that have a three- to five-year Erne horizon and address specific technology challenges that 
are important to the U.S. Department of Defense— its customer. One of the key reasons DARPA works well is because it clearly understands 
its customer's needs; the U.S. Department of Defense is a consumer of these research projects. As we understand it, if DARPA funds 
something that solves a challenge, the military is nearly certain to adopt it, so the market funcEons well. To put it in economic terms, there is 
a demand from the military (which is funded by the federal government), and the federal government funds the supply side as well. The two 
match up. 

8. In her book, Marguerite Roza shows how districts don’t even realize that their strategy, in pracEce, is very different from what their 
intended one is, as their actual spending and resource allocaEon processes do not align at all to their stated goals. It’s not that the market is 
fragmented per se, but that districts act in seemingly quirky ways and often define the requirements of their jobs to be done differently from 
each other; there are long buying cycles in educaEon; and districts often break contracts when changes occur in their leadership. All of these 
processes make it difficult for innovaEve products and services to penetrate the market. See Marguerite Roza, Educational Economics: Where 
Do School Funds Go? (Washington, D.C.: The Urban InsEtute Press, 2010). 

9. Today, for example, the educaEon software market itself is sorted by features and funcEons (what educaEon technology companies say 
their tools can do), rather than by actual use cases (to what end schools are actually using those tools in particular classrooms, relaEve to 
particular students, with particular needs). The more complete our research cycles become, the more valuable informaEon will be available to 
pracEEoners trying to choose from the vast array of technologies that can best support their students. 

10. For example, the U.S. Department of EducaEon could coordinate an RFP process looking for anomalies to the 2 Sigma study, around 
which many learning technologies loosely base their individualized learning products. It is worth noEng that this type of process would 
obviously require some degree of data transparency Eed to the original research upon which the anomalies are built. 
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11, The total amount of federal dollars allocated specifically to education R&D for fiscal years 2010-15 was $1.14 billion, or 2.85 percent 
of the total U.S. Department of Education budget during that same time period. This number was determined by looking at the amount of 
funds allocated specifically for the R&D function within the Institute of Education Sciences— the statistics, research, and evaluation arm of 
the U.S. Department of Education— as part of the total U.S. Department of Education budget These numbers, shown in the table below, 
were taken from the federal education budgets for fiscal years 2010-15, see "Budget News— U.S. Department of Education,” U.S. Depart- 
ment of Education, http://www2.ed.gov/about/overview/budget/news.html and "Department of Education: Archives of Budget News,” U.S. 
Department of Education, http://www2.ed.gov/offices/OUS/Archives/archive.html (accessed September 9, 2015). 


Federal Education Budget, FY 2010-15 


Fiscal year (FY) 

U.S. Department of 
Education budget 

Institute of 
Education Sciences 
budget 

R&D portion of 
Institute of 
Education Sciences 
budget 

Percentage of U.S. 

Department of 
Education budget 
allocated 
specifically to 

R&D* 

2010 

$59.2 B 

$659 M 

$200.2 M 

3.38% 

2011 

$69.9 B 

$659 M 

$200.2 M 

2.86% 

2012 

$68.1 B 

$593.7 M 

$189.8 M 

2.78% 

2013 

$68.4 B 

$593.7 M 

$189.8 M 

2.78% 

2014 

$67.3 B 

$577 M 

$179.9 M 

2.67% 

2015 

$67.1 B 

$574 M 

$179.9 M 

2.68% 

AVERAGE (2010 - 
15) 

$66.6 B 

$609.4 M 

$190 M 

2.85% 


*These percentages were found by dividing the R&D portion of the Institute of Education Sciences budget by the tota 
U.S. Department of Education budget. 
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